The present invention relates to the motion reconstruction method from inter-frame feature correspondences. More particularly, the invention relates to the method of obtaining 3D motion information from a singular video stream by transforming the similar motions retrieved from a motion library and fitting them into the input image information.
Recently, the production of a virtual character animation using a motion capture technology is becoming very active.
The advantage of using the motion capture technology is that real motions can be produced in a fast and efficient way in comparison to the other conventional animation production methods.
Since the captured motion clips are short and generally related to particular characters or environments, there has been a continues development of various types of editing tools which recycle the captured motions for the production of new animations.
Based on these developed tools, animators can appropriately use the captured motions according to various requirement of the virtual characters and environments.
The monocular images captured by a camera is the most standard medium for storing the motions of a human and to date many researchers have been conducting experiments for various purposes in order to extract human motions from the monocular images.
There has been continues research activities on automatic reconstruction of motions from images based on an image analysis technology. In general, these researches rely on a probabilistic model for ascertaining the position of an articulated body.
Among the examples, Azarbayejani, et al (C. Wren, A. Azarbayejani, T. Darrel and A. Pentland. Pfinder: Real-time Tracking of the human body. IEEE Trans. Pattern Analysis and Machine Intelligence, 1991) proposed a method of real time tracking of a human body from the images obtained from one or plurality of cameras.
The above paper classifies a human body into a number of blobs and 3D location of a blob is tracked by the probabilistic model.
Bregler and Malik (C. Bregler and J. Malik. Estimation and tracking kinematics chains. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1998.) expressed the kinematics of a multi-articulated body in terms of a twist and exponential basis. Based on this expression, the motions of a human body are inferenced from the images obtained from one or plurality of cameras.
Sminchisescu and Triggs (C. Sminchisescu and B. Triggs. Covariance scaled sampling for monocular 3D body tracking. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition, 2001.) brought the attention to the difficulty of reconstructing images based 3D information due to the vagueness and occlusion. They attempted to obtain a nearest solution using an optimization method.
In order to avoid local minimum solutions during an optimization, the paper uses the covariance-scaled sampling method in conjunction with a numerical optimization method.
Also, it concentrated on some effective methods which are already known to some researchers for extracting a previously known 3D information from 3D feature information.
For example, Zheng and Suezaki (J. Y. Zheng and S. Suezaki. A model based approach in extracting and generating human motion. Proceedings of Fouteenth International Conference on Pattern Recognition, 1998.) proposed a model based method of capturing the motions of a multi-articulated body from monocular images.
The above paper disclosed the production method of 3D motions from total images by designating a number of key frames, obtaining 3D information from the key frames and storing them.
Rehg, et al (J. M. Rehg and T. Kanade. Visual tracking of high DOF articulated structures: an application to human hand tracking. European Conf. on Computer Vision, 1994.) attempted to reconstruct 3D information using a probabilistic approach that includes a kinematics model and limiting conditions of articulation angle as well as other limiting conditions.
Kakadiaris and Metaxas (I. Kakadiaris and D. Metaxas. Model-based estimation of 3D human motion with occlusion based on active multi-viewpoint selection. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1996.) disclosed the method of obtaining a ratio for the given model from one piece of image using anthropmetry information.
Taylor (C. J. Taylor. Reconstruction of articulated objects from point correspondences in a single uncalibrated image. Computer Vision and Image Understanding, 2000.) disclosed the method of obtaining a detailed 3D information based on a previously known model using the foreshortening phenomenon which occurs from an image.
Liebowitz and Carlsson (D. Liebowitz and S. Carlsson. Uncalibrated motion capture exploiting articulated structure constraints. In Proc. 8th International Conference on Computer Vision, 2001.) disclosed the method of obtaining a detailed dynamic information of a multi-articulated body based on the images obtained from a plurality of uncalibrated cameras.
The above paper uses a limiting condition which states that the body ratio of a multi-articulated body is constant with respect to time.
Recently, a number of fresh attempts have appeared which shed a new light into the problem of motion construction.
For example, Howe, et al (N. R. Howe, M. E. Leventon, and W. T. Freeman. Bayesian reconstruction of 3D human motion from single-camera video. Cambridge Research Laboratory TR-CRL-99-37, 1999.) attempted to solve the problem of reconstructing 3D motions from monocular images using the relationship between 2D features formed by training and 3D positions.
The above paper claimed that the loss of depth information can be reproduced by using the above relationships.
Sidenbladh, et al (H. Sidenbladh, M. J. Black, and D. J. Fleet. Stochastic tracking of 3D human figures using 2D image motion. European Conference on Computer Vision, 2000.) obtained the patterns of human's walking motion through a training. By using these patterns, an attempted has made to reconstruct an arbitrary walking motion.
The common characteristic for this type of problems is that it regards the 3D motion tracking problems as an inference problem and approached them accordingly.